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Infeasible-interior-point paths are the main 
tools in interior-point methods for solving 
many kinds of optimization problems. 
These paths are usually parametrized by a 
penalty-parameter r iO and further param- 
eters describing their off-centrality and 
infeasiblilty. Starting with an early result 
of C. Witzgall et al. [12] in linear pro- 
gramming, this paper gives an overview 
on results concerning the existence of 
these paths, their analyticity and the limit- 
ing behavior of their derivatives as r i 0, 
and this also for degenerate problems in 
the areas of linear programming, linear 
complementarity problems, and semi- 
defmte programming. 
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1. Introduction 

This paper deals with the analysis of certain paths 
that are the main tool in sequential unconstrained opti- 
mization techniques (SUMT) and interior point meth- 
ods. These were first studied systematically by Fiacco 
and McCormick in [3] as a means to solve constrained 
minimization problems of the form 



inf{/(u)|g,(ti)<0,/=l,2,...,m}. 



(1) 



In particular, they used Frisch's logarithmic penalty 
function (see Frisch [4], [5]) and studied paths u{r), r > 
0, of local minimizers of the penahzed problem 

inf{/(w)-r^log(-gX^))l giiu) <0, /=l,2,...,m} 

as r vl 0. They investigated the local existence and 
uniqueness of these paths, their differentiability proper- 



ties for r > and the convergence of both u{r) and u\r) 
for r sL 0, if r > is small and u{r) is close to a local 
optimal solution u of problem (1). They used the cru- 
cial assumption that u satisfies the standard regularity 
conditions, as the existence of Lagrange multipliers, the 
first order necessary and the second order sufficiency 
condition for a local minimum, and the strict comple- 
mentarity condition. These assumptions are natural and 
they allow the use of the implicit function theorem to 
analyze the path. However they also exclude degener- 
ate problems of practical importance: These occur e.g. 
already with convex and even linear or quadratic prob- 
lems (1) that have more than one optimal solution u or 
which, together with their optimal Lagrange multipli- 
ers, violate strict complementarity. 

Another limitation of paths like u{r) is that they exist 
only for strictly feasible optimization problems (1). 
More general are infeasible-interior-point paths that 
use strictly feasible solutions of a family of suitably 
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perturbed optimization problems that shrink to the 
unperturbed problem (1) as the path parameter r tends 
to zero. Again this poses the question of the existence 
of such paths and their limiting behavior, also for 
degenerate unperturbed problems. 

It is the purpose of this paper to give an overview 
over some results that are relevant in this context. 

In a joint paper [12] with P. T. Boggs and P. D. 
Domich, C. Witzgall was one of the first to deal also 
with degenerate problems in the context of linear pro- 
grams. Their pioneering results will be outlined in Sec. 
2. Similar and even sharper results have been found for 
much more general classes of programs. We describe 
some of them that refer to linear complementarity prob- 
lems in Sec. 3. Still more difficult are semidefinite lin- 
ear programs and semidefinite linear complementarity 
problems. Results for them are described in the final 
Sec. 4. 



u : = lim uir) e Sr. , u\0) : = lim 

ao ^ ^ ^ ^ ^ ao r 



The limit point u is the analytic center of Sjy, and the 
function u{r) is analytic in rfor r > 0. 
Let 

y{r)'=c-A^u{r\ x.(r) :=^--, ? = 1,...,«. 

Then (x, y, u){r), r > 0, the primal-dual central path, 
solves 



iPD\ 



Ax = b, X > 0, 
A^u -\- y = c, y>0, 
X ° y = re, 



where e is the vector 



2. Linear Programs 

Consider the dual linear programs 

(P) min c^x (D) max b^ u 



x: Ax = b, 
x>0, 



y,u: A^u+y = c, 



where A = (a^ ..., aj is a real m x n-matrix of rank m 
with columns a^. Let Sp and Sj^ be the optimal solution 
sets of (P) and (D). 

Consider for (D) the Frisch logarithmic penalty func- 
tion and its minimum u(r) for r > 0, 

n 

u(r) :=argmin{-Z)^w-rV log(c. - af u)\ jfu< c}. 



The path {w(r)}^>o? if it exists, is called the dual central 
path\ then u = u(r) solves the equation 



& = rX 



=1 c^ - a^ u 



(2) 



A central result of [12] adapted to {P) and (D) is 

Theorem 1 Assume 

(a) rank A = m, 

(b) (D) is strictly feasible, 

(c) Sf, is nonempty and bounded. 

Then the path u(r) is well-defined for r > and the fol- 
lowing limits exist: 



e: = (hh...,lfeR\ 
and X ° J is the componentwise product 

By Eq. 2 the primal-dual path exists in the situation of 
Theorem 1. In general, the primal-dual path is well- 
defined for r > 0, provided (P) and (D) are strictly fea- 
sible; primal-dual interior point methods try to follow 
this path for r -l 0. But there are some difficulties with 
this concept: 

- (PD)^ is a nonlinear equation for each r, so one can 
at best approximate points on it. 

- The path does not exist, if (P) and (D) are not 
strictly feasible, even when they are solvable. 

These difficulties are avoided if one uses infeasible 
(weighted) interior-point paths instead, defined as the 
solution z : = (x, y, u) = z(r, vj) = (x, y, u)(r, Tj) of the 
nonlinear system 



(PDX, 



Ax = b-\- rb,x>0, 
A^u + y = c + re, y > 0, 
x° y = rVj, 



where r > 0, < ?] e i?" is a weight vector, and Z?, c are 
suitable perturbation vectors. The infeasible interior- 
point central path belongs to the weight vector e. 

As initialization one may choose Tq : = 1, any Xq > 0, 
Jo > 0, Uq, and put 

b:=AxQ-b, c:=A\-\-yQ-c, ?7o:=^o°Jo- 
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Then (xq, y^, Uq) = (x, y, u)(rQ, rjo) solves at least {PD)^^ 
for r = Tq and 7] = rJQ. The choice Xq = yQ: = e leads to 
rJQ = e, that is to a point on the central path. 

The practical significance of these paths lies in their 
use in infeasible-interior-point methods (see e.g. 
Mizuno [10] for an early analysis of such methods in 
the context of linear programming). Typically these 
methods start with an initial point 

zo = z(ro, ?7o). ^0 > 0, ?7o > 0, 

on the initial trajectory at (fq, t\^ defined by the initial- 
ization. Then these methods construct further iterates 



z^ = z{r^ TIj:), r^>Q, < % 



R' 



k=h2,..., 



where the r^^ > r^^+i > • • • > converge to and the 
weights T}}^~e stay bounded and close to e. The con- 
struction of Zjt+i from Zf^ use, as a rule, the tangent of 
z(., rji^) at Fj^ that is the derivative z^ : = dz(r, r}i^)/dr\^^ 
(usually called the affine scaling direction). The quali- 
ty of these methods depends on how quickly the rf^ con- 
verge to 0. With most methods, the superlinear conver- 
gence of the F/^ relies on the existence (and bounded- 
ness) of the derivatives z^ = z' (r^^, r}^) as r^^ i 0. 

Also with the paths z(r, rj) the same questions arise 
as before: 

- their existence and uniqueness for < r < Tq = 1 , ?] 
>0, 

- their analytical properties, 

- and their limiting properties for r i 0. 

We will answer these questions in the context of 
more general complementarity problems in Sec. 3. 
Here, we only formulate linear programs as particular 
complementarity problems, which is well-known. 

By the duality theorem of linear programming, feasi- 
ble solutions X, (y, u) of (P) and (D) satisfy 

0<c^x- b^u = {A^u+ yYx - {AxYu = y'^x, 

and they are optimal solutions of {P) and (D), if they 
are feasible and x^y = 0. 

By elimination of u, define the linear manifold M a 

M = {(x,y) \3u: Ax = b,A% -\-y = c}, 
and the associated linear subspace 

<D: = {(x, y) \3u : Ax = 0, A'^u -\- y = 0}, 



Then 

(x, y)e O => x^3^ = 0, 

and by the assumption rank (A) = m, 

dimO = dim5?l^= n. 

The linear programs (P) and (D) are then equivalent 
to the associated complementarity problem of finding a 
solution (x, y) of 

(x,y)e M, 
(LCP,p) x>0,j>0, 

xo y = Q (<^x^y = 0). 

Under the assumptions of Theorem 1 , this particular 
complementarity problem satisfies 

(a) dimO = n 

(b) x^y = for all (x, y) e O (3) 

(c) (LCPlp) is solvable. 

3, General Linear Complementarity 
Problems 

A general (TCP) (in horizontal form) is to solve 
(LCP) 



Px + Qy = q, 

x>0,y>0, 

X o J = (<^ x^j = 0), 



where q e R"" and P, Q are real n x n-matrices with rank 
[P,Q] = n. With respect to (LCP) we define: 

1 . z = (x, j) is a (strictly) feasible point of (LCP), iff 
Px-\- Qy = q and z>0 (resp. z > 0). 

2. The set of solutions of (LCP) is denoted by S. A 
solution (x, y) e S is called strictly complementa- 
ry, ifx + J > 0. 

3. O c R^" is the linear space 

O : = mP.Q] = {(x.y) I Px+Qy = 0}, 

4. (LCP), resp. O, is called monotone, if 

(x, y)e O => x^3^ > 0. 

The LCPs associated with linear programs are mon- 
otone; by Eq. 3 they even satisfy x^y = for all (x, y) e 
O. 
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For the presentation of the following results, we 
make the following rather weak standard assumption 

1) rank [P,Q] = n, i.e. dimO = n, 
{V) 2) (LCP) is monotone, 

3) 5';^0, (ZCP)issolvable. 

Note that (F) 3) does not require the unique solvabil- 
ity of {LCP). Degenerate problems with more than one 
solution are not excluded. 

Still more general than monotone complementarity 
problems are sufficient linear complementarity prob- 
lems. They were introduced by Cottle, Pang, and 
Venkateswaran in [2] and further characterized by 
Valiaho [11]. They are the most general class of linear 
complementarity problems of the form (LCP) with the 
following properties: 

1. For all g, the solution set S = S(q) of (LCP) is a 
convex (perhaps empty) set. 

2. Each stationary solution of the (nonconvex) quad- 
ratic optimization problem 

inf {x^ I (x, 3^) is a feasible point of (LCP)} 

associated with (LCP) is a solution of (LCP). 

Again, infeasible (weighted) interior-point paths 
play an important role in methods for solving (LCP). 
They are defined as the solutions (x, 3^) = (x, y)(r, rj) of 



(LCP),, 



Px-\-Qy = q-\-rq, (x, 3^) > 0, 



where r > 0, < 77 e 7?" is a weight, and g e 7?" is a per- 
turbation vector. 

For initialization, choose any Xq > 0, Jq ^ 0? ^0 ^ ^•> 
and put 

q : = (Pxo + Qyo - qyro, ??o : = ^o ° yo/ro- 

Then at least (xq, 3^0) = (x, y)(ro, rjo), and 

qeM,^:={(Px-^Qy-q)/r,\x>0,y>0}. 

Interior-point methods for solving (LCP) try to follow 
such paths for r i in much the same way as described 
for linear programs in Sec. 2. 

As before, the same questions with respect to these 
paths (x, y)(r, rj) arise, 

- their existence and uniqueness for < r < Tq = 1, 
1>0, 



- their analytical properties as functions of r, 77, 

- and their limiting properties for r l- 0. 

The following theorem (see Stoer and Wechs [9]) can 
be viewed as a generalization of Theorem 1 , the result 
ofWitzgalletal.: 

Theorem 2 Assume (V\ Tq > 0, and q e M^. Then the 
following holds: 

a) The path z(r, rf) : = (x, y)(K rf) is a well-defined 
analytic function of(r, rj) for all (r, rf) e (0, Tq] x 
RI+, which converges for r l- to a solution of 
(LCP): 

3z*(77) = (x*,/)(77) : = lim(x, j;)(r,77) e S. 

b) If in addition, (LCP) has a strictly complementa- 
ry solution, then there is an£>0, so that the func- 
tion z(r, TJ) can be extended to an analytic func- 
tion z(r, 7]) on the enlarged domain of all (r, rf) e 
[-£, To] X R%. 

c) If (LCP) is solvable but has no strictly comple- 
mentary solution, then there is an £> 0, and an 
analytic function z(p, rj) on the domain (p, rj) e 
[-£, ^] X 7?% so that z(p\ 7]) = z(p, ri)for all 

(p,Tl)e(0,^,]xRl. 

("(x, y)(r, rj) is an analytic function of p = 
Vr even in r = 0.") 

3.1 Remarks 

1. By a result of Goldman and Tucker [6], any solv- 
able complementarity problem associated with 
linear programs has also a strictly complementary 
solution. So, part b) of the theorem applies in this 
case, which generalizes 

Theorem 1 : the limits 

z*^^>(77):=lim|^z(r.77), ^ = 0,1,..., 

of all derivatives exist, and are in fact the deriva- 
tives of an analytic function z(r, 77) at r = 
extending z(r, rj). 

2. It is easy to find convex quadratic programs for 
which the corresponding complementarity prob- 
lem is only solvable but has no strictly comple- 
mentary solution, so case c) of the theorem can 
occur. 

3. Theorem 2 remains true for sufficient comple- 
mentarity problems (i.e. if "monotone" is replace 
by "sufficienf in Assumption (V) (see [8]). 
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The proof of Theorem 2 is not easy in the case of 
degeneracies, when (LCP) has more than one solution 
or no strictly complementary solution. The mathemati- 
cal reason is that the solutions (x, y, r, ?]) of {LCP),.^ 
with x>0,>'>0, r>0 and ?] > then form a semialge- 
braic variety with a singularity in r = 0, so that the 
implicit function theorem cannot be applied directly 
(though indirectly see [9], [8] for details). 



4. Semidefinite Linear Programs and 
Complementarity Problems 

In this section we show that the results of Theorems 
1 and 2 even generalize to semidefinite linear programs 
and to semidefinite linear complementarity problems. 

We use the following notation, which is standard: 

y : = {^ e if""" \A=A'} is the space of all real 
symmetric n x n-matrices. 

^ is provided with the scalar product {A,B) : = 
tr {B'^A) = tr (BA). 

AyO (resp. ^ :^ 0) means that A e S" is positive 
semidefinite (resp. positive definite), and ^ is the 
set of all positive definite matrices in ^ . 

Then the standard dual pair of semidefinite linear 
programs has the form 

(P) inf (QX) 

X: (A, X) = b,,i= 1,2, „„m, 
XhO, 



(D) 



Here, 



sup b^u 



C, A, e y for / = 1, ..., m, b = (b,, ..., bj ^ ^^ 

The following results are known from the duality 
theory of semidefinite linear programs: 

If (P) and (D) have strictly feasible solutions then they 
also have optimal solutions, and X and (u, Y) are opti- 
mal for (P) and (Z)), respectively, iff 



{A.,X) = b^J = l,2,„„m, 

m 

XY = o,xyo,Yyo. 

By elimination of u one can formulate this as a par- 
ticular semidefinite linear complementarity problem, 
which in general has the following form: Solve 

(SDLCP) P(X) + Q(Y) = q, 

XY=o,x, yyo. 

Here, qe R\n:=n(n+ l)/2 = dim^, and P, Q: ^ -> 
R" are linear operators of the form 

PiX) = i(P^, X), ..., (P„, X)Y, P, e y, 
QiX) = i{Q„ X), ..., <a, JOY, Qi e ^. 

As in the previous section we define: 

1 . The nullspace ^{iP,Q^ of \P,Q\ is denoted by 

o : = {(X, y) I P{X) + e(y) = o,x, fg y}. 

2. (SDLCP), resp. O, is called monotone, if 

(X,Y)e^=>{X,Y)> 0. 

3. The set <F of feasible points of (SDLCP) is defined 
by 

<F:={(XY)\ P(X) + Q(Y) = q,X, YhO}, 

4. The set of solutions of SDLCP is denoted by 

S: = {(X,Y)E <F\XY=0}. 

5. A solution (X,Y)eS is called strictly complemen- 
tary if X+YyQ, 

In the case of the dual semidefinite linear programs 
(P), (D), O is given by 

O = {(X,7) I (4,X) =0, for / =],..., m,7 = 
-^u.A. for some u}, 

and O even satisfies 
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so the corresponding (SDLCP) is monotone. It is not 
known whether in this case the corresponding (SDLCP) 
always has a strictly complementary solution if it is 
solvable. 

The following standard hypothesis, which corre- 
sponds to the standard assumption of the previous sec- 
tion, is very weak 

1) {P(X) + Q(Y)\X,Ye ^}=R\ 
(V) 2) (SDLCP) is monotone, 

3) S^Q, (SDLCP) is solvable. 

The definition of infeasible weighted interior-point 
paths differs slightly from the previous section: For < 
r < Tq and weight matrices M >- 0, infeasible weighted 
interior point paths are now defined as solutions (X, 
Y) = (X,Y)(r,M) of 

P(X) + Q(Y) = q + rq, X>0,Y>0 
(XY+YX)/2 = rM. 

Here, the condition J^= of (SDLCP) is weakened to 
(XY + YX)/2 = rM, which corresponds to the "AHO" 
approach proposed by Alizadeh, Haeberly and Overton 
in[l]. 

IhQ perturbation vector q is chosen such that q e M,^ 
for some Tq > 0, where 



^,\Pix)±M)zl\ 



XJ,XY + YX>0 



A possible initialization is to choose Tq : = 1, Xq = 
7o : =/andMo \ = I ^ndq \ = P(X^) + Q(Y^) - q. 

Again the same questions arise for these paths (X, 
Y)(r, M) as in the previous sections, namely their 

- existence and uniqueness for < r < Tq, M >- 0, 

- analytical properties as fiinctions of r and M, 

- and their limiting properties for r J' 0. 

The following is the main result on (SDLCP). It cor- 
responds to Theorems 1 and 2, and shows the fmitful- 
ness of the early results of C. Witzgall: 

Theorem 3 Assume (F), Tq > and q e IM^^. Then: 
L The path function (XY)(r, M) is a well-defined 
analytic fiinction for all (r, M) e (0, Tq] x ^, 
which converges to a solution of (SDLCP) as r i 
0, 

3(X%r)(M):=lim(X,y)(r,M)e ^. 

riO 



2. If (SDLCP) even has a strictly complementary 
solution, then there is an £> so that the path 
function (X, Y)(r, M) can be extended to an ana- 
lytic function (X,f)(r, M) defined on the enlarged 
domain (r, M) e (-£, r ] x ^^. 
"(X, Y)(r, M) is analytic in (r, M) even at r = 0." 

The proof of this theorem is even more involved than 
that of Theorem 2, it can be found in Preiss and Stoer 

m. 

It is an open problem, whether (and how) the path 
fiinction (X, Y)(r, M) can be extended, if (SDLCP) sat- 
isfies only (V), but has no strictly complementary solu- 
tions. 



5. References 

[1] F. Alizadeh, J.-P. Haeberly, and M. Overton, Primal-Dual 
Interior-Point Methods for Semidefinite Progranoming: 
Convergence Rates, Stability and Numerical results, SIAM J. 
Optim. 8, 746-768 (1998). 

[2] R. W. Cottle, J.-S. Pang, and V. Venkateswaran, Sufficient 
Matrices and the Linear Complementarity Problem, Linear 
Algebra AppL 114/115, 231-249 (1989). 

[3] A. V. Fiacco and G. P.McCormick, Nonlinear Programming: 
Sequential Unconstrained Minimization Techniques, 
JohnWiley and Sons, Inc. (1968). 

[4] K. R. Frisch, Principles of Linear Programming — With 
Particular References to the Double Gradient Form of the 
Logarithmic Potential Method, Memorandum of October 18, 
1954, University Institute of Economics, Oslo. 

[5] K. R. Frisch, The Logarithmic Potential Method of Convex 
Progranoming, Memorandum of May 13, 1955, University 
Institute of Economics, Oslo. 

[6] A. J. Goldman and A. W. Tucker, Theory of Linear 
Progranoming, in Annals of Mathematical Studies 38, Linear 
Inequalities and Related Systems, H. W. Kuhn and A. W. 
Tucker, eds., North Holland, Amsterdam (1956) pp. 53-97. 

[7] M PreiB and J. Stoer, Analysis of Infeasible-Interior-Point Paths 
Arising with Semidefmite Linear Complementarity Problems, 
Math. Programming Ser. A 99, 499-520 (2004). 

[8] J. Stoer and M. Wechs, Infeasible-Interior-Point Paths for suffi- 
cient Linear Complementarity Problems and Analyticity, Math. 
Programming 83, 407-423 (1998). 

[9] J. Stoer and M.Wechs, On the Analyticity Properties of 
Infeasible-Interior-Point Paths for Monotone Linear 
Complementarity Problems, Numer. Math. 81, 631-645 (1999). 
[10] S. Mizuno, Polynomiality of Infeasible-Interior-Point 
Algorithms for Linear Programming, Mathematical 
Programming 67, 109-119 (1994). 
[11] H. Valiaho, P*-Matrices are just Sufficient, Linear Algebra 

Appl. 239, 103-108(1996). 
[12] C. Witzgall, P. T. Boggs, and P. D. Domich, On the Convergence 
Behavior of Trajectories for Linear Programming, in 
Contemporary Mathematics 114, Mathematical Developments 
Arising from Linear Programming Algorithms, J. C. Lagarias 
and M. J. Todd, eds., American Mathematical Society, 
Providence, RI (1991) pp. 161-187. 



140 



Volume 111, Number 2, March- April 2006 

Journal of Research of the National Institute of Standards and Technology 



About the author: Josef Stoer is a Professor Emeritus 
at the University of WUrzburg, Institute of Applied 
Mathematics and Statistics. 



141 



